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This study examined the effects of the Local Systemic Change Through Teacher Enhancement 
Initiative (LSC) on 1,596 in-service high school mathematics teachers’ knowledge. Because data 
are clustered by schools, a hierarchical linear model was calculated. It was found that five of 
the characteristics of the professional development, time to reflect, support to implement, hours 
attended, assistance to implement from LSC, and participation in message boards, had 
statistically significant effects on increasing teacher knowledge, but time to work with other 
teachers, race, educational background, and years taught did not have a statistically significant 


effect. 
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Introduction 

There are many different ideas about what makes professional development effective, but 
there is some agreement on what key features are important (Borko, 2004; Darling-Hammond, 
Wei, Andree, Richardson, & Orphanos, 2009; Desimone, 2009; Garet, Porter, Desimone, 
Birman, & Yoon, 2001; Heck, Banilower, Weiss, & Rosenberg, 2008). Desimone (2009) argues 
for her Core Conceptual Framework with five key features of professional development that are 
content focus, active learning, coherence, duration, and collective participation, and that these 
key features lead to an increase in teacher knowledge. In the Eisenhower professional 
development programs, Garet et al. (2001) found that focus on content knowledge, active 
learning, and coherence were all statistically significant predictors of enhanced knowledge and 
skills when controlling for sponsor, traditional or reform, span, and contact hours. 

Between 1997 and 2006, the National Science Foundation funded 86 professional 
development projects, 48 of which were for math teachers, across the country through the Local 
Systemic Change Through Teacher Enhancement Initiative (LSC) (Heck et al., 2008). According 
to Heck et al. (2008), the main goals of the LSC projects were to include all teachers in targeted 
schools with a minimum of 130 hours of professional development and to implement LSC- 
designed curriculum materials in the teachers' classrooms. The individual projects were designed 
to include key features of effective professional development, but the main focus was to increase 
teachers’ content knowledge, pedagogical content knowledge, and the use of investigative 
pedagogical strategies (Heck et al., 2008). To assess these outcomes, teachers were expected to 
complete a questionnaire at the end of a year of participation in the LSC project (Heck et al., 
2008). 


Research Question 
What factors of an LSC professional development for high school math teachers predict 
perceived increase in knowledge by the teacher? 
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Method 
Data 

The data was collected from the National Science Foundation's Local Systemic Change 
through Teacher Enhancement Initiative (LSC). This project offered professional development 
for K-8 and 6-12 teachers in mathematics and science over the school years 1996-97 to 2005-06 
(Banilower, Heck, & Weiss, 2007; Heck et al., 2008). Each year of the project, principals and 
teachers were required to complete a Principal Questionnaire or Teacher Questionnaire, with a 
response rate of 80% expected for the teachers and 90% expected for the principals (Heck et al., 
2008). Several of the questions in the questionnaires changed in the initial two years, but the 
questions stabilized by the third year (Heck et al., 2008). For questions that were changed, data 
from later questionnaires were both recoded to the original scale and included in the new scale 
with missing values for the original questionnaires. The data set is available by request from 
Horizon Research. 

Participants 

The full data set included over 80,000 participants’ responses to the four teacher 
questionnaires, K-8 math, 6-12 math, K-8 science, and 6-12 science, but the data were filtered to 
focus just on the 9-12 math teachers which left 3,064 of the full data set. Removing any 
participants who were missing data for the variables of interest left 1,596 teachers in the sample 
data to be analyzed. 

The participants are all 9-12 mathematics teachers who took part in LSC professional 
development. The participants are split almost evenly between males (45%) and females (55%). 
The participants are primarily white (80%), with other races under 10% (black). For educational 
background, the majority had an undergraduate major in mathematics or mathematics education 
(60%) and certification to teach (60%). Additionally, a large percentage held a graduate degree 
in mathematics or mathematics education (38%). On average, the participants had 12.30 years of 
teaching experience. 


Table 1: Descriptive Statistics of Teacher Control Variables by Sample and Original Data 


Sample Data Original Data 
M SD | Min | Max N M SD | Min | Max N 

Male 43 50| 0 1 | 1,596 45 50) 0 1 3,023 
Hispanic .03 17| 0 1 | 1,596 .03 18) 0 1 3,064 
Indian 01 11 0 1 | 1,596 01 11 0 1 3,064 
Asian .02 15 0 1 | 1,596 .03 17] 0 1 3,064 
Black .09 29| 0 1 | 1,596 10 31) 0 1 3,064 
Pacific Islander .00 04] 0 1 | 1,596 .00 .03 | 0 1 3,064 
White 83 37| 0 1 | 1,596 .80 40) 0 1 3,064 
No Race Indicated .02 14] 0 1 | 1,596 .03 17] 0 1 3,064 
Undergraduate Major 61 49 | 0 1 | 1,596 .60 49) 0 1 3,020 
Undergraduate Minor 10 30] 0 1 | 1,596 10 30) O 1 3,020 
Graduate Major or Minor 38 49 0 1 | 1,596 38 49 | O 1 3,020 
Teaching Certificate 63 48 0 1 | 1,596 .60 49) 0 1 3,020 
None of the Above .07 .26| 0 1 | 1,596 .08 27) 0 1 3,020 
Education 

Years Taught 11.86 | 7.67 1 21 | 1,596 12.30 | 7.79 1 21 | 3,038 


Note. N = 3,064 for the original data and N = 1,596 for the sample data. Years taught is on a 
scale of 1 to 21+ years, with the maximum coded as 21. For all other variables, 0 indicates "no" 
and | indicates "yes." 
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For the original data set, the data are nested within 19 professional development projects with 
a range of 1 to 400 participants in each project. There mean number of teachers per project is 
161.26 with a standard deviation of 117.94. For the sample data, the data are nested within 18 
professional development projects with a range of 3 to 223 participants in each project. There are 
a mean number of 88.67 teachers per project with a standard deviation of 61.74. 


Table 2: Descriptive Statistics by Project ID by Sample and Original Data 


Sample Data Original Data 
Project ID | Count | Percent Count | Percent 
04 56 Pel 98 3.20 
09 fe: 4.82 153 4.99 
10 93 5.83 188 6.14 
18 223. |) 1397 400 | 13.05 
19 13 81 16 oe 
28 0 .00 1 .03 
31 51 3.20 124 4.05 
36 98 6.14 195 6.36 
39 129 8.08 249 8.13 
40 58 3.63 101 3.30 
42 153 9.59 224 il 
44 95 D935 232 1d 
48 68 4.26 123 4.37 
58 144 9.02 284 De, 
71 183 | 11.47 369 | 12.04 
2 3 19 8 .26 
a 4 20 11 36 
74 120 (eee 249 TA5 
78 28 Ls 58 1.89 


Note. N = 3,064 for the original data and N = 1,596 for the sample data. 


Measures 

The control teacher variables come primarily from the teacher questionnaire. The variables 
for gender and race are dummy variables created from participants choosing one of a list of 
choices, as seen in Table 1. Participants also chose from a list of educational backgrounds, but 
were allowed to choose all choices that applied to them, as seen in Table 1. For years taught, 
participants chose from a list with ranges of years taught (0-2, 3-5, 6-10, 11-15, 16-20, 21-25, 26 
or more). These were initially coded as dummy variables, but were operationalized as the mean 
of each range to turn them into more meaningful continuous variables. 

Most of the covariate measures also come directly from the teacher questionnaire. These 
include the amount of time to work with other teachers during the professional development, 
time to reflect during professional development, and support to implement what was learned in 
the professional development. All three of these variables came directly from the questionnaire, 
with a rank of 1 to 5, with | labeled as "Not at all" and 5 labeled as "To a great extent." The 
number of hours of professional development attended was listed in 11 categories on the 
questionnaire (0, 1-9, 10-19, 20-39, 40-59, 60-79, 80-99, 100-129, 130-159, 160-199, and 200 or 
greater). These were initially coded as dummy variables from 1 to 11, but were operationalized 
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as the mean of each range to change them into continuous variables with more meaning in the 
context of the research question. 

The final three professional development variables, assistance to implement the professional 
development from LSC, participation in the LSC message boards, and increase in knowledge 
from professional development, are each means of multiple questions from the questionnaire. All 
of the original questions for these measures are on the same scale as the number of times 
participants did each activity (0, 1-2, 3-4, 5-6, and 7 or more). Assistance to implement the 
professional development is the average of four sources of assistance, coaching by LSC staff 
based on observations, LSC teacher leaders, LSC district staff, or LSC mathematicians or 
mathematics educators, with a = .7247 with all four measures included. Participation with 
message boards was the average of three sources of participation, reading messages, posting 
messages, and discussion groups, but the a = .5107 with all three sources and a = .6440 with 
just reading messages and posting messages. Thus, participation is only the average of the two 
sources that leave the highest a, reading and posting on message boards. Perceived increase in 
knowledge was initially the average of three sources of knowledge, mathematics content 
knowledge, understanding of how children think/learn about mathematics, and ability to 
implement high-quality mathematics instructional materials, with a = .8887. However, a = 
.8907 by removing mathematics content knowledge, so perceived increase in knowledge is just 
the average of the two measures of teaching knowledge. 


Table 3: Descriptive Statistics of Professional Development Variables by Sample and 


Original Data 
Sample Data Original Data 
M SD_ | Min | Max| N M SD_ | Min | Max| N 


Amount of time to | 2.84 | 1.23 1 5 | 1,596 2.81 | 1.22 1 5 1,962 
work with other 
teachers 
Level of time to 2.76 | 1.19 1 5 | 1,596 Q73-\ 118 1 a, 1 | L057 
reflect 
Level of support 3.42 | 1.15 1 5 | 1,596 3204S 1 Sy | 1952 
to implement what 
was learned 
Hours attended 93.78 | 67.04 | 5 | 200 | 1,596 62.07 | 69.32 | O | 200 | 2,843 
Level of assistance | 1.43 | 1.62 0 7 | 1,596 1.40 | 1.60 0 7 1,908 
to implement from 
LSC 

Amount of 1.25 | 1.40 1 6 | 1,596 1.25 | 1.40 | 0 7 | 1,919 
participation in 
message boards 
Perceived increase | 3.32 | 1.09 1 5 | 1,596 3.32 | 1.10 1 5 1,941 
in knowledge 


Note. N = 3,064 for the original data and N = 1,596 for the sample data. 


Analysis 
Given that the original analysis was concerned with how the teachers were able to implement 
the LSC curriculum materials in their classrooms, the possible covariates and control variables 
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were limited by the data collected in the teacher questionnaire. The covariates used were amount 
of time to work with other teachers during professional development, level of time to reflect 
during professional development, level of support to implement what was learned in professional 
development, hours of professional development attended, level of assistance to implement 
professional development from LSC, and amount of participation in LSC message boards. 
Research has shown collective participation (Desimone, 2009; Franke, Carpenter, Levi, & 
Fennema, 2001; Garet et al., 2001) and duration (Banilower, Heck, & Weiss, 2007; Garet et al., 
2001; Heck et al., 2008) are both predictors of teachers' gains in knowledge from professional 
development. Other research has shown that aid to implement what is learned in professional 
development is related to teachers’ long term use of what was learned in the professional 
development (Carpenter, Fennema, & Franke, 1996; Fennema et al., 1996; Franke et al., 2001). 
Participation in online forms of professional development has also been shown to be effective in 
helping teachers to gain knowledge (Boling & Martin, 2005; Dede, Breit, Ketelhut, McCloskey, 
& Whitehouse, 2005; Herrington, Herrington, Hoban, & Reid, 2009). The possible teacher 
control variables were gender, race, educational background, and number of years teaching. All 
were included, because it is likely that each will have an influence on how teachers view what 
they have learned from the professional development projects. 


Results 

The sample data are nested in 18 LSC professional development projects, so a hierarchical 
linear regression model was calculated. This model produced similar results to the multiple 
regression model, with the same professional development covariates being statistically 
significant predictors of perceived increase in knowledge from the professional development. In 
model 2, these variables include time to reflect on what was learned in the professional 
development, support to implement what was learned in the professional development, hours of 
professional development attended, assistance to implement what was learned in the professional 
development from LSC, and participation in LSC message boards; all were statistically 
significant at p < .001. Unlike the multiple regression however, the only statistically significant 
teacher control variable was gender (t(1595) = —6.00; p < .001). 
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Variables and Teacher Control Variables 


Model 1 Model 2 

Perceived increase | Perceived increase 

in knowledge in knowledge 
Time to work with other .0147 .0101 
teachers 
Time to reflect 09358*** A081 *** 
Support to implement 2067*** AST TTe* 
Hours attended .0047*** .0045*** 
Assistance to implement A124? OTT ET? 
from LSC 
Participation in message .0683*** 791 *=* 
boards 
Male = 2719*** 
Hispanic .2645 
Indian -.1649 
Asian -.1416 
Black .0763 
Pacific Islander -.8439 
No Race Indicated -.0513 
Undergraduate Major 0191 
Undergraduate Minor .0754 
Graduate Major or Minor .0109 
Certification to Teach .1726 
Years Taught -.0074 
Constant 1.6898*** 1.8256*** 
N 1596 1596 
Note. t statistics in parentheses, *** p < .001, ** p < .01,* p < .05. 
Reference group is white for race and no education in mathematics or 
mathematics education. 
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Table 4: Hierarchal Linear Model of Increase in Knowledge on Professional Development 


Both the multiple regression and the hierarchical linear model agree on the factors of the 
LSC professional development that related to the teachers' perceived increase in knowledge. The 
number of hours of professional development attended has the greatest standardized coefficient, 
so it has the greatest impact on the projected increase in knowledge. Specifically, for every one 
standard deviation increase in number of hours attended by a teacher, there is a projected .2889 
standard deviation increase in knowledge. This agrees with the research findings that say that the 
duration of a professional development is one of its key features (Desimone, 2009; Garet et al., 
2001; Heck et al., 2008). Another result that agrees with research is that the support and the 
assistance to implement the professional development is related to the increase in knowledge. 
Franke et al. (2001) found that the teachers felt the support of the professional development team 
was a critical factor in their ability to sustain what they learned in the professional development. 
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Finally, participation in the message board was also a statistically significant predictor of the 
teachers’ increase in knowledge, which agrees with research that online professional 
development can help teachers increase their knowledge (Boling & Martin, 2005; Dede et al., 
2005; Herrington et al., 2009). 

One result that does not agree with research is about the time to work with other teachers. 
Research shows that collective participation is a key feature of professional development 
(Desimone, 2009) with a positive relationship to increased knowledge (Garet et al., 2001). 
However, the multiple regression found working with other teachers to have a negative 
relationship that was not a statistically significant predictor of increase in knowledge. 
Limitations 

These results are based on self-reported survey data from the participants about their views 
on the professional development project and what they learned from the professional 
development project. However, Desimone (2009) argues that surveys are a reliable data source 
for behavior-based constructs about frequencies of events, such as teachers' experiences in 
professional development and experiences implementing the professional development. This data 
is a mix of behavior-based questions and evaluative questions, so using a survey may not be the 
most reliable source of data collection. Another limitation is that the teachers are rating their 
own perceived increase in knowledge, rather than using a pretest and posttest of knowledge to 
quantify how much teachers learned. This makes it difficult to know how much the teachers 
actually learned from attending the professional development. 

Next Steps 

This analysis focused on teacher knowledge, but not on how teacher knowledge affects 
teaching practices or student achievement. Research has shown that teacher knowledge has an 
impact on both practice in the classroom and on students' achievement (Ball, Thames, & Phelps, 
2008; Hill, Rowan, & Ball, 2005; Kennedy, 1999). Although data is not available about student 
achievement for the LSC projects, the teacher questionnaire included questions about teaching 
practice. Thus, a next step would be to analyze whether the change in knowledge predicts 
teaching practice, as was found in other research on professional development (Desimone, 2009; 
Garet et al., 2001; Heck et al., 2008). 
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